This invention relates to a moving picture edit processing for providing moving pictures by shooting, recording and editing them with a video camera, etc., and more particularly, this invention relates to a moving picture playback apparatus for playing back and displaying moving pictures, and a moving picture collection apparatus for collecting moving pictures.
As a general and conventional moving picture playback apparatus for playing back moving pictures, there is a video tape recorder (VTR). With VTR, moving pictures are recorded on a video tape. When a user presses buttons such as xe2x80x9cplaybackxe2x80x9d, xe2x80x9cstopxe2x80x9d, xe2x80x9cframe forwardxe2x80x9d, xe2x80x9cframe rewindxe2x80x9d etc., on the VTR apparatus, which the video tape is set to, the tape is controlled according to an instruction, and the recorded moving pictures are displayed on a screen.
Furthermore, as the general and conventional moving picture collection apparatus for collecting moving pictures, there is a video camera recorder. The video camera recorder is a video camera combined with the VTR in one piece so as to make it easy to carry, and moving pictures shot with the video camera are to be recorded on the video tape by the VTR.
Here, let us consider a case when we shoot moving pictures with the video camera, and record and edit the moving pictures and produce a video title.
On the video tape, several scenes, for example, a xe2x80x9cScene at the Airportxe2x80x9d, a xe2x80x9cScene at the Portxe2x80x9d, a xe2x80x9cScene on a Superhighwayxe2x80x9d, a xe2x80x9cScene on the Railroadxe2x80x9d, etc., are recorded. Suppose if we are to produce a video title regarding ground transportation, we have to select appropriate scenes such as the scenes at the airport, the scenes on the superhighway, the scenes on the railroad, etc., from the video tape, and then rearrange and concatenate them in a desired order. Moreover, in order to playback a certain segment of a scene, we have to set a playback segment in the scene, and we have to implement processing so as to specify the playback speed in order to playback the part in slow motion. For additional effects, such as overlapping the scene of the railroad on the scene of the superhighway, several scenes must be composited.
As an example of moving picture edit apparatus designed for such editing based on the moving pictures recorded on the video tape, there is a video edit system which combines the edit control apparatus with two or more VTRs. This system designates one VTR as an input VTR and the other VTRs as output VTRs, and edit operation such as processing or concatenating scenes are carried out by dubbing necessary scenes from the input VTR to output VTR in the order. Also, a composite edit operation is implemented by compositing signals from several input VTRs by the edit control unit and then sending out to the output VTRs.
Those are examples of the conventional and general moving picture playback apparatus and the moving picture edit apparatus using it, in recent years, a moving picture edit apparatus using a computer called a non-linear edit system is widely put to use (a typical product of the non-linear edit system is the MCXpress by Avid Technology Company of the United States). The non-linear edit system applies the technology which digitallizes moving picture data, compresses the data, and stores it in a computer memory, and then reads the data as necessary and playbacks after expanding the data. When this non-linear edit system is compared with the previously mentioned video edit system, the non-linear edit system has an advantage in editing without repeating dubbing from the video tape to the video tape, and thereby it""s usage is gradually spreading.
The operation of the non-linear edit system, which is one example of the conventional moving picture edit apparatus, will be explained next.
FIG. 38 shows a general configuration of the conventional non-linear edit system.
The non-linear edit system is configured with a sequential access type moving picture storage unit 2 (the moving picture storage unit for VTR, etc., and called as the sequential access type moving picture storage unit hereinafter in order to distinguish it from the moving picture storage unit using a hard disk of a computer.) connected to a computer 50. In a typical configuration of the non-linear edit system, computer 50 and sequential access type moving picture storage unit 2 are connected with a signal cable and control cable. The signal cable inputs signals of moving pictures or sound played back by sequential access type moving picture storage unit 2 into computer 50. And the control cable is used for controlling playback, stop, frame forward, or frame rewind from computer 50 to sequential access type moving picture storage unit 2, and for transmitting the frame number of the frame currently displayed from sequential access type moving picture storage unit 2 to computer 50.
Moving pictures shot with the video camera is recorded on a medium such as the video tape, and then stored in the sequential access type moving picture storage unit 2.
The user performs editing by inputting a command from a command input unit 8 using a keyboard or a mouse of the computer.
An example of user interface for editing is illustrated in FIG. 39.
In the figure, there are shown a scene menu window 111, a displayed scene icon 112, a moving picture playback window 113 for playing back one of the scenes, playback control button 114 for controlling moving picture playback by the moving picture playback window 113, a stop button 115, a playback button 116, a frame rewind button 117, a frame forward button 118, a scene definition window 119 for specifying a starting frame and an ending frame for each scene, a new scene display button 120 for displaying new scenes, and a moving picture edit window 121 for concatenating and compositing each scene and defining a new scene.
The editing is performed in the following procedure using the user interface aforementioned.
(1) Defining a Scene
The user firstly defines a scene of related segment in the moving pictures such as the scene of the airport, the scene of the superhighway, and so forth, by a scene definition unit 20. The information of the defined scene is stored in scene information storage unit 5. The scene information contains a specified starting frame number and ending frame number for each scene.
FIG. 40 indicates one example of the scene information.
The explanation of the frame number is now given. A moving picture, for instance, consists of 30 images per second. Each image is referred to as the frame. Moving pictures are recorded on a medium such as the video tape by sequential access type moving picture storage unit 2, and the number specifying each frame on the medium is called the frame number.
Business VTRs use the number called the time code for the purpose of specifying frames. The time code is expressed with an 8-digit number such as 01:10:35:08, which represents the 8th frame of one hour, ten minute, and 35 seconds.
Defining a scene is, for example, carried out as below.
First of all, in FIG. 39, a new scene display button 120 is clicked with the mouse. Then, a new scene having no defined title, starting frame and ending frame is displayed on moving picture playback window 113. The scene is then defined by inputting the title, starting frame number, and ending frame number at the keyboard on scene definition window 119.
For defining a scene in the previously mentioned method, the user has to know the starting frame number and the ending frame number of the scene to be defined beforehand. Business VTRs are provided with a function to display the time code of the frame currently displayed on the monitor screen, which enables the user to know the starting frame number and the ending frame number by reading the displayed time code from the screen by positioning the desired frame using such buttons as the forward button, rewind button, frame forward, frame rewind, etc., of the VTR.
(2) Reading a Moving Picture
Then, the defined scene is digitally compressed by a moving picture digital compression unit 6, and stored in direct access type moving picture storage unit 7 of computer 50 (a hard disk memory, and is referred to as the direct access type moving picture storage unit hereinafter to distinguish it from sequential access type moving picture storage unit 2 of the VTR, etc.).
With regard to the format of digital compression of moving pictures, there are various types of formats such as MPEG, Motion-JPEG, Cinepak, and so forth, but because the computation amount becomes huge whichever system is used for digital compression, it is common to execute by inserting an extension board special for digital compression into the computer so as to shorten the processing time.
The processing for digital compression of moving pictures becomes as explained below.
The system first performs processing of digital compression for each scene defined by scene definition unit 20. Because each scene has its starting frame and ending frame described, computer 50 controls sequential access type moving picture storage unit 2 to position the starting frame of a target scene.
Then, the playback of sequential access type moving picture storage unit 2 is started, and at the same time, an instruction is sent to moving picture digital compression unit 6, and the processing for digital compression starts. Moving picture digital compression unit 6 inputs signals output from sequential access type moving picture storage unit 2, and converts each frame into digital data, and performs data compression. The obtained digital compression data is stored in direct access type moving picture storage unit 7.
(3) Displaying a Scene
In FIG. 39, scene menu window 111, for each scene stored in a scene information storage unit 5, moving picture display unit 9 creates a scene icon 112 and displays. For scene icon 112, any image which allows the user to easily grasp the content of the scene can be used, but it is typically created by contracting the first frame of each scene. Furthermore, when one of the scenes is selected by the click of the mouse, moving picture display unit 9 displays the selected scene on a moving picture playback window 113. The display of moving picture playback window 113 can be controlled by playback control button 114, and the playback of the moving pictures is started when playback button 116 is clicked, and the playback of the moving pictures is stopped when stop button 115 is clicked. Or, when frame rewind button 117 or frame forward button 118 is clicked, a frame immediately before or after the current frame is displayed and stopped.
In such a case, moving picture display unit 9 reads the moving picture data digitally compressed and stored in direct access type moving picture storage unit 7, expands the data on the spot, and performs processing of displaying the obtained data on the screen.
The edit operation explained below is proceeded by displaying and checking the contents of each scene as thus far explained.
(4) Executing an Edit Operation
The edit operation involves processing, concatenating, and compositing scenes. As an example of a scene processing, there are a setting of a playback area or partial change in playback speed. This operation is performed by specifying the playback area by the frame number on scene definition window 119. Also, a scene concatenation or scene composition is performed by lining each scene icon 112 on moving picture edit window 121. On moving picture edit window 121, the horizontal axis corresponds to the time axis, and when scene icon 112 is lined within this window in order, each scene is concatenated from left to right in order and the new scene is defined. Moreover, there are two tracks A and B on moving picture edit window 121, and when scenes are lined at the same time belt on both tracks, two scenes are composited.
The defined scene is stored in direct access type moving picture storage unit 7, and the scene information is stored in scene information storage unit 5. An example of describing the newly defined scene is shown in FIG. 41.
(5) Creating a Moving Picture
The new scene is defined by the above mentioned edit operation. Based on the description of the scene, moving picture creation unit 11 reads the moving picture data of each scene stored in direct access type moving picture storage unit 7, and processes, concatenates, composites, and performs other operations to create a new moving picture data. The created moving picture data is again stored in direct access type moving picture storage unit 7. When the moving picture data is read and copied onto a medium such as a CD ROM, the medium can be distributed and data can be played back on other computers. It is also possible that the moving picture data is played back and the signal can be transmitted to sequential access type moving picture storage unit 2, and then recorded onto the medium such as the video tape, then the medium can be distributed and the data can be played back on the VTR, and so on.
The conventional moving picture playback apparatus and the moving picture collection apparatus are configured as above mentioned. Let us suppose now that one hundred scenes are shot with the video camera, and each scene is about 30 seconds, and then edit those moving pictures. In order to perform editing, scenes in the moving pictures must first be defined. In order to define the scenes, a work of specifying the starting frame and ending frame for each of the 100 scenes must be performed. For defining one scene, if it takes about 30 seconds to perform the operation for finding out the starting fame and ending frame by implementing the forward, rewind, frame forward, frame rewind, etc. of the VTR, then, it amounts to about 50 minutes to define 100 scenes.
Also, for the processing of compressing data by digitallizing the moving pictures, because it generally takes about the same time period as the continuation time period of the moving pictures, it takes about 50 minutes for processing digitally compressing 100 scenes and storing in the direct access type moving picture storage unit.
In this way, the conventional moving picture playback apparatus and the moving picture collection apparatus pose a problem of taking substantial time period until the edit operation is enabled after the shooting of moving pictures has ended, in order to define the scenes or to read the moving pictures.
Or, in defining the scene, the work of specifying the starting frame and ending frame of the scene exactly is a troublesome task which cannot be done without careful attention.
Furthermore, the moving picture data must be digitally compressed and stored in the computer memory, if, for instance, data compression is performed in the MPEG1 compression format, the digitally compressed moving picture data becomes about 1.5 megabit data amount per second. In order to digitallize all 100 scenes and store in the computer memory in the MPEG1 format, a storage capacity of at least 500 mega bytes is required.
Also, in order to perform digital compression of moving pictures, an expansion board special for digital compression of moving pictures must be inserted into the computer.
In this way, because the conventional moving picture edit apparatus needs a large capacity storage or a compression board special for moving pictures, it poses problems that it becomes a large-size apparatus not suited to carry around or for use in a limited space such as a home or office.
This invention is designed to solve above-mentioned problems, and it aims at obtaining a moving picture playback apparatus or a moving picture collection apparatus which can define scenes or read moving pictures in a short time, and confirm the contents of the collected moving pictures quickly.
The first aspect of the invention is a moving picture playback apparatus which comprises
a moving picture storage unit for storing a moving picture shot by a shooting device;
an event detection unit for detecting an event of one of a change in a shooting subject, the change in the shooting device, and an operation of a shooter during shooting, while shooting with the shooting device;
a scene definition unit for defining a segment, as a scene, separated by a frame which is a screen of the moving picture corresponding to an event and by a frame corresponding to a next event based on the event detected by the event detection unit;
a scene information storage unit for storing scene information of the scene defined by the scene definition unit;
a command input unit for inputting an instruction from a user; and
a moving picture display unit for reading and displaying the moving picture stored in the moving picture storage unit based on the instruction input by the command input unit and the scene information stored in the scene information storage unit.
The second aspect of the invention is the moving picture playback apparatus, wherein the event detection unit may detect, as the event, one of the change in a position of the shooting device, direction of the shooting device, and a lens condition during shooting.
The third aspect of the invention is the moving picture playback apparatus, wherein the event detection unit may detect, as the event, one of an operation of a scene start and an operation of a scene end of the shooter during shooting.
The fourth aspect of the invention is the moving picture playback apparatus which may further comprise:
an edit unit for reading the moving picture stored in the moving picture storage unit based on the instruction input by the command input unit and the scene information stored in the scene information storage unit, and processing, compositing, and concatenating the moving pictures to edit a new scene, and storing scene information of the new scene in the scene information storage unit; and
a moving picture creation unit for creating a moving picture for the new scene based on the scene information of the new scene stored in the scene information storage unit.
The fifth aspect of the invention is a moving picture playback apparatus which may comprise:
a moving picture storage unit for storing a plurality of moving picture;
a scene definition unit for defining a segment in the moving picture stored in the moving picture storage unit as a scene;
a scene information storage unit for storing scene information of the scene defined by the scene definition unit;
a typical frame storage unit for storing a typical frame which represents the scene defined by the scene definition unit;
a typical frame determination unit for selecting the typical frame, which represents the scene, in the scene defined by the scene definition unit, reading the selected typical frame from the moving picture storage unit, and storing in the typical frame storage unit;
a command input unit for inputting an instruction from a user; and
a moving picture display unit for displaying the typical frame stored in the typical frame storage unit based on the instruction input by the command input unit.
The sixth aspect of the invention is the moving picture playback apparatus, wherein the moving picture display unit may display the typical frame from the typical frame storage unit if the frame to be displayed is the typical frame, and display the closest typical frame before the typical frame to be displayed from the typical frame storage unit if the frame to be displayed is not the typical frame.
The seventh aspect of the invention is the moving picture playback apparatus, wherein the moving picture display unit may display the frame to be displayed from the moving picture storage unit if the frame to be displayed is not the typical frame when the instruction input from the command input unit may be one of a stop, frame forward, and frame rewind.
The eighth aspect of the invention is the moving picture playback apparatus, wherein the moving picture display unit may display the typical frame from the typical frame storage unit if the frame to be displayed may be the typical frame when the instruction input from the command input unit may be one of the stop, frame forward, and frame rewind.
The ninth aspect of the invention is the moving picture playback apparatus may further comprise an event detection unit for detecting an event of one of a change in a shooting subject, the change in a shooting device, and an operation by a shooter during shooting with the shooting device, and
wherein the typical frame determination unit may select a frame corresponding to the event detected by the event detection unit as the typical frame of the scene including the frame.
The tenth aspect of the invention is the moving picture playback apparatus may further comprise an edit unit for reading the moving picture stored in the moving picture storage unit, process, composite, and concatenate the moving pictures based on the instruction input by the command input unit and the typical frame stored in the typical frame storage unit, and edit a new scene, and store scene information of the new scene in the scene information storage unit, and a moving picture creation unit for creating a moving picture for the new scene based on the scene information of the new scene stored in the scene information storage unit.
The eleventh aspect of the invention is a moving picture collection apparatus may comprise:
an event detection unit for detecting an event of one of a change in a shooting subject, the change in a shooting device, and an operation of a user during shooting with a shooting device; and
a frame number obtainment unit for obtaining a frame number of a frame which may be one screen of the moving picture corresponding to the event; and
an event information storage unit for storing event information which may correspond to the event detected by the event detection unit and the frame number.
The twelfth aspect of the invention is the moving picture collection apparatus, wherein the event detection unit may detect, as the event, one of the change in a position of the shooting device, the change in a direction of the shooting device, the change in a lens condition, an operation of a recording start of a user and an operation of a recording end of the user.
The thirteenth aspect of the invention is the moving picture collection apparatus, wherein the event information storage unit may obtain an image data of a frame in the moving picture corresponding to the event, and may obtain the change in the position of the shooting device corresponding to the event, direction of the shooting device and the change of the lens condition corresponding to the event, and store as the event information.
The fourteenth aspect of the invention is the moving picture collection apparatus, wherein the event information unit may input and store an explanation sentence added by the user corresponding to the event stored in the event information storage unit.
The fifteenth aspect of the invention is the moving picture collection apparatus may further comprise an explanation sentence input unit for obtaining the image data of the frame in the moving picture corresponding to the event stored in the event information storage unit, displaying the image data of the frame on a display screen, and having the user input an explanation sentence of the event corresponding to the image data of the frame displayed on the display screen, and
wherein the event information storage unit may store the explanation sentence from the explanation sentence input unit.
The sixteenth aspect of the invention is the moving picture collection apparatus, wherein the event detection unit may detect, as the event, an approach of the shooting device toward a preset shooting subject.
The seventeenth aspect of the invention is the moving picture collection apparatus, wherein the event detection unit may detect, as the event, one of an entering, and a coming out of the shooting device to/of a preset shooting area.
The eighteenth aspect of the invention is the moving picture collection apparatus may further comprise a shooting device control unit for sending an order to the shooting device to change its direction toward the shooting subject corresponding to the event of approaching of the shooting device toward the preset shooting subject corresponding to the event detected by the event detection unit.
The nineteenth aspect of the invention is the moving picture collection apparatus, wherein the shooting device control unit may send an order of one of a recording start and a recording end corresponding to the event of one of entering and coming out of the shooting device to/of the preset shooting area.
The twentieth aspect of the invention is the moving picture collection apparatus may further comprise an event information display unit for displaying the event information, a command input unit for inputting an instruction from a user based on the displayed event information, and a moving picture playback instruction unit for selecting the event based on the instruction input by the command input unit, and specifying the frame number of the frame in the moving picture corresponding to the selected event, and instructing a playback of the moving picture from the frame specified by the frame number.
The twenty-first aspect of the invention is a moving picture collection apparatus may comprise:
a moving picture input unit for inputting a moving picture shot by a shooting device;
an event detection unit for detecting an event one of a change in a shooting subject, the change of the shooting device, and an operation by a user during shooting with the shooting device;
an event information storage unit for expressing information of the event detected by the event detection unit with image and sound, and compositing the event information with the moving picture input by the moving picture input unit and storing it.
The twenty-second aspect of a moving picture playback apparatus may comprise
a priority set unit for setting a priority for a frame of a moving picture composed of a plurality of continuous frames;
a priority obtainment unit for obtaining the priority set for the frame of the moving picture by the priority set unit;
a playback speed set unit for setting a moving picture playback speed at which the moving picture is played back;
a frame selection unit for selecting a frame of the moving picture based on the playback speed set by the playback speed set unit and the priority set for the frame of the moving picture obtained by the priority obtainment unit;
a frame obtainment unit for obtaining an image of the frame selected by the frame selection unit; and
a moving picture display unit for displaying the image of the frame obtained by the frame obtainment unit.
The twenty-third aspect of the invention is a moving picture playback apparatus may comprise:
a priority set unit for setting a priority for a frame of a moving picture composed of a plurality of continuous frames;
a frame obtainment unit for obtaining an image of the frame of the moving picture;
a priority obtainment unit for obtaining the priority set for the frame of the moving picture by the priority set unit;
a typical image obtainment unit for obtaining the image of the frame set a high priority by the priority set unit as a typical image;
a typical image storage unit for storing the typical image obtained by the typical image obtainment unit;
a playback speed set unit for setting a moving picture playback speed at which the moving picture is to be played back;
a frame selection unit for selecting the frame of the moving picture based on the playback speed set by the playback speed set unit and the priority set for the moving picture frame obtained by the priority obtainment unit; and
a moving picture display unit for displaying the image of the frame selected by the frame selection unit from one of the frame obtainment unit and the typical image storage unit.
The twenty-fourth aspect of the invention is the moving picture playback apparatus, wherein the moving picture display unit may read and display the image of the frame from the typical image storage unit when the playback speed set by the playback speed set unit may exceed a normal playback speed, and read and display the image of the frame from the frame obtainment unit when the playback speed may be below the normal playback speed.
The twenty-fifth aspect of the invention is the moving picture playback apparatus, wherein the typical image storage unit may change the frame to be stored as the typical image according to a timing relation between the frame of the typical image stored by the typical image storage unit and the frame currently displayed by the moving picture display unit.